On the Importance of Word Sense Disambiguation for Information Retrieval

Research in information retrieval has led to mixed results about the impact of natural language processing. This paper discusses the importance of word sense disambiguation despite these mixed results. We first discuss some of the factors that can cause apparent inconsistency in retrieval performance with regard to natural language processing: instability of test collection queries, different baselines, averaging performance over queries, incorrect linguistic analysis, software errors, frequency of linguistic expressions, and differences in how results are reported. We then focus on the problems with word sense disambiguation in particular and why disambiguation is important for improving retrieval performance. The paper concludes with recommendations for how systems should be evaluated.